Methods and apparatus for enhancing color vision and quantifying color interpretation

ABSTRACT

In one embodiment, a method is disclosed that includes selecting a first color sample within a target area in a first image of a first object displayed by a display device; selecting a second color sample within a target area in a second image of a second object displayed in the display device; comparing the first color sample against the second color sample to determine a measure of color difference or a measure of color equivalence between the first color sample of the first object and the second color sample of the second object; and displaying the results of the comparison to a user in the display device. One or more of these functions may be performed with a processor.

CROSS REFERENCE TO RELATED APPLICATIONS

This non-provisional patent application is a continuation application ofU.S. patent application Ser. No. 16/219,934, filed Dec. 13, 2018, whichis a continuation application of U.S. patent application Ser. No.14/675,719, filed Mar. 31, 2015 (now abandoned), which claims thebenefits of U.S. Provisional Patent Application No. 61/973,208 filedMar. 31, 2014 (now expired).

Furthermore, this application is related to U.S. patent application Ser.No. 14/419,939 entitled METHOD AND APPARATUS FOR PERFORMING ANDQUANTIFYING COLOR CHANGES INDUCED BY SPECIFIC CONCENTRATIONS OFBIOLOGICAL ANALYTES IN AN AUTOMATICALLY CALIBRATED ENVIRONMENT filedFeb. 6, 2015, which is incorporated herein by reference for allpurposes. U.S. patent application Ser. No. 14/419,939 is a nationalphase application claiming priority to International Patent Ap. No.PCT/US2013/035397, filed on Apr. 5, 2013 by Bernard Burg et al.(hereinafter Burg '397), which is incorporated herein by reference forall purposes. This application is also related to U.S. patentapplication Ser. No. 14/633,513 entitled QUANTIFYING COLOR CHANGES OFCHEMICAL TEST PADS INDUCED BY SPECIFIC CONCENTRATIONS OF BIOLOGICALANALYTES UNDER DIFFERENT LIGHTING CONDITIONS filed on Feb. 27, 2015 byBernard Burg et al., which is incorporated herein by reference for allpurposes. U.S. patent application Ser. No. 14/633,513 claims the benefitof U.S. Provisional Pat. Application No. 61/948,536, filed on Mar. 5,2014 by Bernard Burg et al. (hereinafter Burg '536), which isincorporated herein by reference for all purposes.

FIELD

The embodiments generally relate to color quantification.

BACKGROUND

Traditionally, color perception in humans is characterized as the colorresolving capability of an average person. Practically the appearance ofa perceived color can be dramatically affected by the human eye relatedissues and observed scene issues.

About 8% of men and 0.5% of women have some color perception limitation.Protanomaly is a reduction in the ability to perceive red, with the rareprotanopia (1% in men) being the complete failure to see red.Deuteranomaly is the reduced perception of green (5% in men).Tritanomaly, the failure to see blue is extremely rare. Properties ofthe eye and retina incur changes in the color sensitivity also occurwith age, including macular degeneration. Error in the perceived colorof a sample is also exacerbated by the surrounding color. This issometimes called ‘color spreading’ or ‘simultaneous contrast’ and isbased upon the subjective judgment of a color changing with the natureand proximity of other colors. Metamerism is an artifact of theperceived color being assessed from the sum of the differentialintensities in each of the three (or more) receptor sensitivity bands.

The spectrum of the illumination of scenes or objects can have a seriouseffect upon the image detected by the camera. The effects ofillumination differs amongst camera and sensor types. The intensity ofthe illumination, which should naturally be spatially uniform across theentire area of observation, and calibration of test samples and witnesspanels, needs to exceed the noise threshold of the least effective(reflection, scattering, refraction, polarization, etc.) sample. Theangle of the illumination and the viewing angle determine the reflectionof the optical system. The material and textures of the object matter asthe primary measurement is the spectral modulation of the illuminationreferred to as the objects apparent ‘color’. What is desirable to knowis what properties are changed between incidence and emission of thelight. While the texture should be neither too smooth (specular) nor toorough (locally variable on an imaged pixel dimension), it should appearLambertian (same brightness—and color—from all directions).

BRIEF SUMMARY

The disclosed embodiments are summarized by the claims that followbelow. Briefly, the disclosed embodiments relate generally to systemsand methods for detecting the presence or absence of a color in a camerafield and to perform color vision, color recognition and colorcorrections. When in controlled lighting environments, color matchingand color corrections are well mastered. However when operating inuncontrolled lighting environments the operations of performing colormatching and color corrections are significantly more complex. Thedisclosed embodiments can: 1) compare colors under similar lightingconditions, 2) compare perceived colors to reference stored in memoryand, 3) compare perceived colors or color variations to any static orkinetic abstract color models (color trajectories for eachconcentration, color trajectories in time etc.) stored or calculated,and, 4) calibrate and correct colors for different lighting conditions.Specific applications relate to methods for detecting the presence orabsence of colors in color samples. These methods may be utilized byprocessors of head-mounted display devices, for example, to providesolutions to color-related applications. Quantified colors, colormatches, color gradients and color differences displayed by ahead-mounted-display device, for example, can enhance a user's visualcapabilities. The color corrections can follow principles based on thehuman vision, including e.g., gamut and metamerism limitations andcorrections for forms of daltonism; or alternatively can work withabsolute color spaces like RGB, CMYK, Munsell, Pantone, and others thatare independent of human eye properties.

BRIEF DESCRIPTIONS OF THE DRAWINGS

This patent or application file contains at least one drawing executedin color. Copies of this patent or patent application publication withcolor drawing(s) will be provided by the United States Patent andTrademark Office upon request and payment of the necessary fee.

FIGS. 1A-1C illustrate a display device of a head mounted display devicedisplaying first selection user interface windows to select a firstcolor sample of a first object for analysis.

FIGS. 2A-2B illustrate a display device of a head mounted display devicedisplaying second selection user interface windows to select a secondcolor sample of a second object for analysis.

FIG. 2C illustrates the display device of the head mounted displaydevice displaying a comparison user interface window including the firstcolor sample of the first object and the second color sample of thesecond object for comparison.

FIG. 3 illustrates the display device of the head mounted display devicedisplaying a results user interface window including color comparisonresults between the first color sample of the first object and thesecond color sample of the second object.

FIGS. 4A-4B illustrate a zone selection method to select a set of colorsof a set of color samples.

FIG. 4C illustrates a color chart including a table of a plurality ofcolors of color samples from which a set may be selected.

FIG. 4D illustrates the zone selection method being used to select a setof colors of a set of color samples from the color chart shown in FIG.4C.

FIGS. 4E-4G illustrate a recall of a set of memorized colors stored in amemory of the head-mounted display device and selection thereof to forma set of selected colors for further processing.

FIG. 5A illustrates a comparison window displayed by a display device tocompare a color of a single color sample to a set of colors in a set ofcolor samples.

FIG. 5B illustrates a results window displayed by a display device inresponse to the comparison of a color sample to a set of colors in a setof color samples.

FIG. 6 illustrates an exemplary head-mounted display device that candisplay a cross hair and user interface instructions on a displaydevice.

FIG. 7A-7B illustrates an automatic color correction method of acaptured color of a color sample in response to a color reference barand different lighting conditions.

FIG. 8 illustrates a reagent dipstick and a color chart with a set ofreference colors that can be compared by the embodiments to determineanalyte concentration

FIG. 9 illustrates a test paddle with reagent test pads and a colorreference bar that may be used to automatically correct captured colorsof the reagent test pads prior to color comparison with a set of colorcalibration curves to determine analyte concentration.

FIG. 10 illustrates a diagram of augmented reality glasses executing anapplication to extract and enhance images of street signs.

FIG. 11 illustrates a diagram of augmented reality glasses executing anapplication to extract and enhance images of color maps.

FIG. 12 illustrates a diagram of augmented reality glasses executing anapplication to compute and display color gradients of a baked good orcooked food undergoing a baking or cooking process and a color gradientchart for comparison with a color gradient curve.

DETAILED DESCRIPTION

In the following detailed description of the disclosed embodiments,numerous specific details are set forth in order to provide a thoroughunderstanding of the embodiments. However, it will be obvious to oneskilled in the art that the disclosed embodiments may be practicedwithout these specific details. In other instances well known methods,procedures, components, and circuits have not been described in detailso as not to unnecessarily obscure aspects of the disclosed embodiments.

The disclosed embodiments include methods, apparatus and systems ofcolor vision tools. in some embodiments, the color vision tools can beused to detect and quantify color changes induced by specificconcentrations of biological analytes in an automatically calibratedenvironment.

Head-Mounted-Displays

There are several ways to augment the reality of a user's vision. Invideo gaming and virtual reality settings, users may wear an opaquehead-mounted display where the reality is recomposed digitally ondigital display screens, including a visual feed of images captured by acamera with digital additions to augment this reality.

Referring now to FIG. 6 , a perspective view of an exemplaryhead-mounted display device 600 is shown. The head-mounted displaydevice 600 includes a frame 612 with left and right assemblies 617A,617Band a pivotal display boom 614. The pivotal display boom 614 ispivotally coupled to the frame 612 at a pivotal joint 672. The pivotaldisplay boom 614 can be pivoted up out of the way of the left and rightassemblies 617A,617B if desired. The pivotal display boom 614 can beaffixed to either one of the arms 640A or 640B of the frame 612 toposition a display device 654 in the view of one of the user's eyes.

The pivotal display boom 614 includes a processor 650, the displaydevice 654, a camera 626, and an optional sensor 628 coupled to theprocessor by wires or circuitry within the display boom 614. The pivotaldisplay boom 614 may also include a storage device as part of theprocessor 650 or a separate storage device 618, such as a memory device.The storage device 618 stores software and/or firmware instructions forexecution by the processor 650 to provide the user interface and performthe functions of the methods described herein. To obtain a more centralpoint of view, the camera 626 can alternatively be mounted to the frame612, such as on a bridge or near a central portion of the frame 612.

The camera 626 can be used to take picture or record a video at theuser's discretion. The camera 626 can also be used by the device toobtain an image of the user's view of his or her environment to use inimplementing augmented reality functionality. The sensor 628 is a sensorassociated with the camera 626, such as a light sensor for example, thatcan be used by firmware and/or software to improve the quality of theimages captured by the camera 626.

The pivotal display boom 614 may also include a radio 682 to be inwireless communication with a radio 694 of a user input device 690. Theuser input device 690 includes a touchpad 692 with one or more buttonsthat can be selected to control the functions of the head-mounteddisplay device 600. Alternatively or additionally, the pivotal displayboom 614 may include a microphone 680 coupled to the processor 650 toreceive voice commands that are recognizable by the head mounted displaydevice 600. The voice commands are user inputs that are used to controlthe functions of the head-mounted display device 600.

The frame 612 includes a band 613 with temples or arms 640A-640B and acentral portion 631 and a bridge 620. The frame 612 may further includeleft and right rims 630 detachably coupled to the band 613. A bridge arm622 with a pad may be coupled to the rims 630 to support the device onthe nose of a user. The left and right arms 640A-640B mount over a usersleft and right ears to further support the device 600. Band 613 can beconfigured to fit on the head of a user with the central portion 631positioned over the brow of the user and supported in a position thereover by pads 624 that contact the nose of the wearer.

The frame 612 may further include one or more earpieces 646 coupled tothe ends 644 of the temples or arms 640A-640B. A battery (not shown) maybe housed in one or both of the earpieces 646 to provide power to theinternal electrical components of display boom 614. A wire may routedthrough a channel or hollow passage in the arms 640A-640B and centerportion 631 to the display boom 614. A battery may alternatively behoused in the display boom 614 itself to provide power to the electricalcomponents therein and avoid using a wire routing through the frame tothe batteries.

The left and right assemblies 617A,617B may include a lens 613A,613Bmounted in the rims 630. Lens assemblies 617A,617B can attach to centralportion 631 by various snap-fit or press-fit arrangements or can beremoveably affixed using screws or the like.

A portable user interface device 690, such as a smartphone or tabletcomputer, may be in wireless communication with the head-mounted displaydevice 600. The portable user interface device 690 includes a touchpad692 to receive user inputs and control the functions of the head-mounteddisplay device 600.

Some disclosed embodiments provide a method and apparatus to address auser's eye deficiencies and display an augmented reality that includescolor matches, color corrections, and color measurements in ahead-mounted display, such as head-mounted-display device 600 forexample, without disconnecting the user's eyes from its environment.

Comparing Two Colors Under Same Lighting

The application executed by the processor of the head mounted displaydevice can enter into a color comparison mode by means of a touchpadcommand or a voice command.

Referring now to FIG. 1A, a display device 110 in a head-mounted displaydevice, such as the display device 654 of the head mounted displaydevice 600 illustrated in FIG. 6 , displays a crosshair 111 in themiddle of the display screen. A user interface of the head-mounteddisplay device displays instructions 112 on how to the use head-mounteddisplay device. For example, the user interface displays the instruction112 to select a target area of color. A user may move his head and thedisplay device 110 so that the crosshair 111 is aligned over a firstobject of a first color for selection of a first color sample.

In FIG. 1B, the user has moved his head with the head-mounted displaydevice such that the crosshair 111 displayed in the display device 110is aligned over a first object 116 of color, such as a t-shirt forexample. A selection user input, such as a button pressed within a touchpad 692 or a spoken voice recognizable command received by a microphone680, for example, may be used to select and capture the color under thecross hair within a target area.

Reference is now made to FIG. 1C. Within a predetermined target areaunder the cross-hair 111, a first targeted color (first color sample)120 of the first object 116 has been selected by the user. The capturedcolor is temporarily stored in memory. The first targeted color (firstcolor sample) 120 within the target area may be subsequently displayednear an edge of the display device 110, such as shown in FIG. 2A. Astorage user input, such as another voice command or button selectedwithin a touchpad 692, can be used to non-volatilely store the selectedcolor in a storage device, such as the memory 618, so that it can bereused later.

After selection of a first color sample, a second color sample of asecond object of a second color may be selected in order to comparefirst and second colors.

Referring now to FIG. 2A, a second selection user interface window isshown such that a similar selection process can be used to capture asecond target color of a second object for the purpose of comparisonwith the first target color of the first object. The previously selectedtarget color, the first target color 120, is shown displayed near a sideof the selection user interface window on the display screen. Anoperational status 222 is indicated by the user interface in a topportion of the selection user interface window on the display device110. The operational status 222 in FIG. 2A illustrates a compare colorsmode. A user instruction 112 displayed in the selection user interfacewindow by the display device instructs the user to select color as thesecond target color for comparison with the first target color 120.

Referring now to FIG. 2B, the user moves his head with the head-mounteddisplay device such that the crosshair 111 displayed in the secondselection user interface window is aligned over a second object 216 ofcolor, such as pants for example. A selection user input is used toselect and capture the color under the cross hair within a target area.

Referring now to FIG. 2C, after selection of the second targeted color225 of the second object 216, the display device 110 displays acomparison window. Within a predetermined target area under thecross-hair 111, a second targeted color (second sample color) 225 of thesecond object 216 has been selected by the user. For visual comparison,the first target color (first sample color) 120 and the second targetedcolor (second sample color) 225 are displayed in the comparison userinterface window on the display screen. These are the color samples thatare to be compared by the processor. The display device continues todisplay the current operational status 222 of the head-mounted displaydevice in FIG. 2C, a compare colors mode.

The processor of the head-mounted display device, such as processor 650,performs a comparison between the captured colors in the selected firsttargeted color 120 and the selected second targeted color 225.

There are several comparisons that can be made between colors by theprocessor. Values of color are often defined by the chosen color spaceused to represent the visible color range of the electromagneticspectrum.

One common color space is the red-green-blue (RGB) color space that isan additive color space. The RGB color space is defined by the threechromaticities of the red, green, and blue additive primaries, and canproduce any chromaticity that is within a triangle defined by theprimary colors. A complete specification of an RGB color space includesa white point chromaticity and a gamma correction curve. For example, astandard red-green-blue (sRGB) color space has a D65 white point and aCRT Gamut. The RGB (Red Green Blue) range of color space covers afraction of the eye's visible color gamut.

Another common color space is the cyan, magenta, yellow, and key (CMYK)color space that is a subtractive color model, often used in colorprinting, and is also used to describe the printing process itself. Theacronym CMYK refers to the four ink colors that are used in some colorprinting cyan, magenta, yellow, and key (black). Another common colorspace is YCBCR. YCbCr is a family of color spaces used as a part of thecolor image pipeline in video and digital photography systems. Y′ is theluma component and CB and CR are the blue-difference and red-differencechroma components.

Other coordinate values in other color spaces may also be used with theembodiments, such as L*a*b* coordinates in the International Commissionon Illumination (CIE) CIE76 color space, L*a*b* coordinates in theInternational Commission on Illumination (CIE) CIE94 color space; andthree color cones of a long, medium, and short (LMS) wavelengthresponsivity color space.

While these other color spaces may be used, the RGB color space iseasier to use to explain how to make color comparisons. A cameratypically captures RGB color pictures, by superposition of filters tocapture each of the red, green, and glue primary color components.Accordingly, the red, green, and blue color values (as well as othercolor space values) are readily available from an image captured by atypical camera without much conversion, if any. The human eye works inthe RGB color space with R, G and B color rod sensors. Chemical testpads on test paddles and test strips have been developed to beinterpreted by the human eye, as such colors should be differentiated bythe eye despite human perception artifacts. The RGB color gamut or rangecovers a larger zone than CMYK. Hence A larger color gamut allows forbetter color recognition.

Amongst several comparisons that can be made between colors by theprocessor, a comparison between first RGB values of the first targetedcolor 120 and second RGB color values of the second targeted color 225can be performed.

The processor first analyzes the color in each of the selected/capturedfirst targeted color 120 and second targeted color 225. The processor650 determines color values for red (R), green (G), and blue (B) thatcan be additively combined together to make up each color of the firsttargeted color 120 and the second targeted color 225. In accordance withsome embodiments, when a color is viewed by a user it analyzed by aprocessor and identified with the name of the color (color name) beingdisplayed in the user interface window for viewing by the user. Inresponse to the RGB color values determined from the targeted colors,the processor associates a color name with each. For example, theprocessor may associate the color name of orange to the first targetedcolor. In response to the RGB color values determined from the targetedcolors, the processor then compares the RGB color values for the firsttargeted color 120 against the RGB color values of the second targetedcolor 225. The processor can then display the results of the comparisonin a results user interface window on the display device 110 to theuser.

Referring now to FIG. 3 , the display device 110 displays the resultsuser interface window of the color comparison between the selected firsttargeted color 120 and the selected second targeted color 225. Theresults user interface window includes the current operational status, astatus of color comparison results 338. In the results user interfacewindow, the first targeted color 120 of the first object is associatedwith a first color name 331 (orange in this example) and firstred-green-blue (RGB) color levels 332 (R:248, G:146, B:81 in thisexample). Similarly, the second targeted color 225 of the second objectis associated with a second color name 334 (red brick in this example)and second RGB color levels 335 (R:183, B:85, B:79 in this example).

As mentioned herein, a comparison between first RGB values of the firsttargeted color 120 and second RGB color values of the second targetedcolor 225 can be performed by the processor. A difference between theRGB color values is calculated by the processor by subtracting eachrespective red, green, and blue value of the second targeted color 225from each respective red, green, and blue value of the first targetedcolor 120. For example, a blue difference value of two is determined bysubtracting a blue value of seventy-nine of the second targeted colorfrom a blue value of eighty one of the first targeted color.

The results of the difference in RGB values can be displayed in thecomparison results window. In the color comparison results window shownin FIG. 3 , the RGB difference values 336 between the first RGB valuesof the first targeted color 120 and second RGB color values of thesecond targeted color 225 are displayed under an RGB difference legend337.

An exemplary application for color matching/comparison is when a usershops for clothes. A plurality of articles of clothing may be comparedfor color matching/contrasting in the same store under the same lightingconditions. For example, color of a shirt 116 shown in FIG. 1B can becompared or matched/contrasted to the color of pants 216 such as shownin FIG. 2A. The color matching/comparison can also be performed underdifferent lighting conditions. Storage devices, such as a memory, canstore one or more colors of a pre-existing cloth stored in a storagelocation, such as in a cupboard or closet in ones home. A processor canthen compare the stored colors from ones home under one lightingcondition with colors of clothes in a shop under different lightingconditions when shopping for a new article of clothing or cloth.

Another example application of color matching and comparison is in theselection of fruit at a grocery store. Different fruits may exhibitdifferent colors. The same fruit may also exhibit different colors basedon ripeness or age. A fruit of a given color viewed by a user may becompared with stored fruit colors to allow proper selection of fruittype and age or ripeness. The stored fruit colors allow selection offruit of the same color.

Selecting Sets of Colors

In accordance with a number of embodiments, colors can be compared tosets of colors. There are a number of ways of selecting sets of colorsfor comparison.

A first method of set selection of colors is for a user to open a setselection mode in the augmented reality glasses or head mounted displaydevice with a voice command or a click on touchpad of a wireless devicein communication with the augmented reality glasses or head mounteddisplay device . Similar to selecting a first color sample and a secondcolor sample described previously with reference to FIGS. 1A-1C and2A-2B, the user can select two or more colors samples one at a time toform a set of selected color samples. A voice command or a buttonselection/click on touchpad can be used to then close the set ofselected color samples when the user is finished doing so. A subsequentcomparison of the set of selected color samples may be made afterclosing the set.

Referring now to FIGS. 4A-4B, a second method of set selection of colorsis based on a zone selection method using the head-mounted-displaydevice is now described. In FIG. 4A, a plurality of color samples 410are desired to be selected as a set of colors for comparison. A voicecommand/touchpad click causes the augmented reality glasses or headmounted display device to enter into a zone selection mode for selectionof a set of colors. Instructions 443 for performing zone selection of aset of colors are displayed on the display device 110 by the userinterface.

The user is instructed by the instruction 443 to select a set of colors.The user moves the augmented reality glasses or head mounted displaydevice to optically align the crosshair 111 of the display device 110 ata first corner 111A of a selection zone 411, such as the upper left handcorner of the plurality of color samples 410. The user then selects thefirst corner 111A of the selection zone by validating the position witha voice command/touchpad click on a touchpad.

Referring now to FIG. 4B, the user moves the augmented reality glassesor head mounted display device to optically align the crosshair 111 ofthe display device 110 at a second corner 111B of the selection zone411, such as the bottom right corner of the plurality of color samples410. The user then selects the second corner 111B of the selection zone411 by validating its position with a voice command/touchpad click on atouchpad.

FIGS. 4C-4D illustrate an exemplary application of the zone selectionmethod of a set of color samples. FIG. 4C illustrates an interpretationtable 412 of urinalysis as provided by manufacturers. The interpretationtable 412 includes a set 414 of a plurality of color samples. Thecomplete set 414 of the plurality of color samples in the interpretationtable 412 can be selected as the set of color samples by the augmentedreality glasses or head mounted display device using the zone selectionmethod.

The user moves the augmented reality glasses or head mounted displaydevice to optically align the crosshair 111 of the display device 110 ata first corner 111A of a selection zone 415, such as the upper left handcorner of the plurality of color samples 414. The user then selects thefirst corner 111A of the selection zone by validating the position witha voice command/touchpad click on a touchpad. The user moves theaugmented reality glasses or head mounted display device to opticallyalign the crosshair 111 of the display device 110 at a second corner111B of the selection zone 415, such as the bottom right corner of theplurality of color samples 414. The user then selects the second corner111B of the selection zone 414 by validating its position with a voicecommand/touchpad click on a touchpad.

Referring now to FIGS. 4E-4G, a third method of set selection of colorsamples using the head-mounted-display device is now described. Colorsamples of color may be selected one at a time and stored in memory ofthe head-mounted display device , such as memory 618 of the head-mounteddisplay device 600 shown in FIG. 6 . Individual color samples of colormay be stored through voice command/touchpad click on touchpad during aselection process, such as the color sample selection process describedherein with reference to FIGS. 1A-1C.

The stored color samples of color can be recalled from the memory of thehead-mounted display device for display by the display device 110. Theuser issues a voice command/touchpad click on a touchpad cause the userinterface of the glasses to enter into a stored color selection mode torecall the stored color samples from memory for display.

Referring now to FIG. 4E, a stored set 440 of color samples of color aredisplayed by the display device 110 in response to the command to enterinto the stored color selection mode. The stored set 440 are displayedin an array of spaced apart color samples.

The user moves the cross-hair 111 on top of the desired color samples ofcolor that are desired to be in the selected set of colors. For example,the cross-hair 111 is moved to be on top of the color sample 444 at aposition 111C. The cross-hair 111 may be moved by motion of the glassesor head mounted display device with known motion control techniques(e.g., using of accelerometers, gyroscopes, compass, eye motiondetection) or by known remote mouse or joy stick control via a touchpad,or voice commands (e.g., move up, move left, stop).

With the cross-hair over the color sample, the color sample may bechosen for inclusion within the set of selected colors. The selectionmay be made by voice command with a microphone or a touchpad click on atouchpad as described herein. The process of moving the cross hair overon top of the desired samples and selecting them for inclusion into theset of color may be repeated over and over again until all desiredcolors of color samples are selected from those stored. A voice commandor a button selection/click on touchpad can then be used to then closethe set of selected color samples when the user is finished doing so.

Reference is now made to FIG. 4G. After the set of selected colors isclosed by the user, the user interface displays the selected set 449 ofcolors color samples selected by the user. The user interface displays astatus 448 for the window that indicates the selected set of colors isbeing displayed. Subsequently, a comparison of the set of selected colorsamples may be made.

Comparing Sets of Colors Under Same Lighting Conditions

Referring now to FIG. 5A, a color comparison may be made between aselected color sample 550 and a selected set 551 of colors. The colorsample 550 may be selected by a similar selection process to thatdescribed with reference to FIGS. 1A-1C. The selected set 551 of colorsamples may be selected by one of the methods described herein withreference to FIGS. 4A-4G. After the selected set 551 of colors isselected, the user may issue a command to a processor, such as in theglasses or head-mounted display device, to enter a comparison mode.

In FIG. 5A, the display device 110 displays a comparison windowincluding the color sample 550 near an edge of the window and the set551 of colors located near a center of the window to provide a visualcomparison of what is to be compared. The user interface generates astatus indicator 553 of “Compare to Set of Colors” on the display device110. A processor, such as processor 650 shown in FIG. 6 , executesinstructions to perform a color comparison between the color sample 550and the set 551 of colors. Upon completion of the comparison, theprocessor can cause the display device 110 to display a results windowindicating a closest match between the color of the color sample 550 andthe color of one of the color samples in the set 551 of color samples.

In FIG. 5B, a results window is illustrated including a user interfacegenerated status indicator 559 of color comparison results. The resultswindow displays the color sample 550 and its associated RGB values 556.The results window highlights the closest matching color in the set 551to the color of the color sample. The results window displays a circleor bulls eye (cross-hair in a circle) 570 about the color sample 555 inthe set 551 of color samples that most closely matches the color of thecolor sample 550.

The RGB values 557 associated with the most closely matched color sample555 are also displayed in the results window in line with the RGB values556 of the color sample 550 for a visual comparison. An RBG difference558 between the RGB values 556 of the color sample 550 and the RGBvalues 557 associated with the most closely matched color sample 555 mayalso be calculated by the processor and displayed in the results windowby the display device 110.

After the user has adequately viewed the results screen, he/she mayissue another command to the glasses or heads mounted display device toreturn to a normal viewing mode, to shut down or turn off, or to performanother selection and color compare process.

There are a number of applications for the color set comparison process.One application for color set comparison is in the medical field and theinterpretation of a urinalysis. Another application for the color setcomparison is to match a paint color to a color in a set of colorsdisplayed on pantone cards.

The color set comparison process may be performed under the same orsimilar lighting conditions. Alternatively, the color set comparisonprocess may be performed under different lighting conditions. That is,the color sample 550 may be illuminated by a first lighting conditionwhile the set 551 is illuminated by a second lighting condition thatdiffers from the first. A color set comparison process performed withsimilar lighting conditions has less noise and therefor can be accuratewithout any calibration or special techniques. It may be desirable in acolor set comparison process with different lighting conditions for thecolor sample and the set of colors to perform further techniques toimprove the comparison results.

Calibrating Lighting and Illuminance

Lighting calibration is often useful for operations involving lightingcorrections, color corrections, as well as mapping colors into acalibrated environment.

One simple way of doing is to look directly into the light source with ahead-mounted-display. The camera in the head mounted display device cangenerate a spectrogram of the captured light from the light source andcan compare it to the type of light associated with pre-existinglighting standards, such as the International Commission on Illumination(CIE) standard illuminants A, B, C, D50, D55, D65, D75, E, and F1-F6.CIE standard illuminant A is intended to represent typical, domestic,tungsten-filament lighting. Illuminants B and C are daylight simulators.Illuminant B serves as a representative of noon sunlight, with acorrelated color temperature (CCT) of 4874 K, while illuminant Crepresents average day light with a CCT of 6774 K. The D series ofilluminants are constructed to represent natural daylight. Illuminant Eis an equal-energy radiator; it has a constant spectral powerdistribution inside the visible spectrum. The F series of illuminantsrepresent various types of fluorescent lighting.

Illuminance can be directly estimated by a camera, such as camera 626shown in FIG. 6 . A rough value of illuminance is reported in the camerametadata (e.g., jpegMetadata.DigitalCamera.BrightnessValue). This roughvalue of illuminance can be used in the process of comparing colors toimprove accuracy.

If lighting conditions of an object is near to one of these lightingstandards, the captured images can be labeled with the type of lightingsource associated with the lighting standard. A comparison of types oflighting sources can be made when it is associated with the capturedimages. A comparison of types of lighting sources may be made todetermine if two color images were captured with the same or differentlighting conditions. If the lighting sources and thereby lightingconditions differ, color corrections may be made to one or more imagesto compensate for the different lighting sources and conditions.

Automatic Color Corrections

Referring now to FIG. 7A, a method of automatic color correction is nowdescribed. The color of a color sample 773A is desired to be recognized.Included adjacent the sample 773 is a color reference bar 770. The colorreference bar 770 may include color samples for one or more of thefollowing colors:—Cyan, Magenta, Yellow, Key (black), Gray, White, Red,Green, Blue.

An unknown lighting source, such as ambient light, is projected onto thecolor reference bar 770 and the color sample 773 to respectively reflectcolors 770A and color 773A back to a head mounted display device 600. Acolor camera (e.g., charge coupled device (CCD), CMOS-sensor) in thehead mounted display device 600 is used to capture a color image of thecolor 773 of the color sample 773 and the colors 770A of the colorreference bar 770 under an unknown light source (e.g., ambient light)with the same lighting conditions for each.

Previously, reflected colors 770B of the color reference bar 770 weredetermined under a known lighting source providing known lightingconditions (e.g., CIE illuminant D65). One way of measuring the colors770B of the color reference bar 770 is by using a spectrophotometer 777with a lighting source providing a known lighting condition (e.g., CIEilluminant D65). The color reference bar 770 is placed under thespectrophotometer 777 and a sensor captures the colors 770B reflectedback from it that were generated by the incident light of a controlledlighting source generating a know type of light source and known type oflighting condition. The known type of lighting condition may beassociated with the colors 770B captured from the reflections on thecolor reference bar 770.

The method of automatic color calibration calculates 772 an inversetransformation (in the form of an inverse transform matrix 774) linkingthe color 770A generated by the color reference bar 770 under an unknownlight condition to the color 770B generated by the color reference bar770 under a known light condition and light source. An inverse transformmatrix 774 is calculated linking the color 770A reflected under anunknown lighting condition to the color 770B reflected under thestandardized D65 light source and conditions that were used in thespectrophotometer 777.

The inverse transform matrix 774 may be used by a processor toautomatically apply 775 a color calibration to the color 773A of thecolor sample 773 determined under unknown light conditions. When thematrix 774 is applied, the color 773A of the color sample 773 underunknown lighting conditions is corrected to the color 773B (anequivalent corrected color) of the color sample 773 under the knownlighting conditions, such as illuminant D65 of the spectrophotometer777.

By incorporating such a reference color bar 700 into the camera field ofthe color sample 773, automatic color corrections, as well as lightingcorrections and color comparisons in an automatically calibratedenvironment, can be performed by methods and apparatus disclosed herein.

Further detailed principles of color correction are described in Burg'397 and incorporated herein by reference.

Comparing Two Colors in Automatically Calibrated Environment

Referring now to FIG. 7B, when a color reference bar 770 is viewed by ahead-mounted-display device 600, a processor can automatically makecolor corrections using an inverse transform matrix 774 to the color773A of a color sample 773 captured under unknown lighting conditions.Oftentimes known colors 783 of known color samples 780 are captured in acalibrated environment under known lighting conditions by aspectrophotometer 777 just as the colors 770B of the color reference bar770. A more accurate comparison 790 may then be made between thetransformed color 773B of the color sample 773 and the known color 783of the known sample 780.

There are a number of applications of automated color calibration withcolor comparisons. To compare or match colors of clothes, images ofcolor samples of clothes or cloth stored in a cupboard or closet underknown lighting conditions may be captured by the heads on display devicein a calibration mode and then compared with automatically correctedcolors of images of color samples captured at a clothing shop underunknown lighting conditions.

The selection of the right skin tone for make-up is complex since everymake-up manufacturer has a different color system. To compare or matchskin tone color of makeup, images of color samples of skin tone withmake up captured under known lighting conditions (e.g., at home) may becaptured by the heads on display device in a calibration mode and thencompared with automatically corrected colors of images of color samplesof skin tone with makeup captured at a shop (e.g., a department store)under unknown lighting conditions. Using the head-mounted display deviceto compare skin tone of makeup under a known lighting (the reference orcalibrated skin tone) with skin tone of makeup applied and sold in shopsunder whatever lighting conditions, overcomes the use of the differentmanufactures of makeup using different color systems.

Additionally, automated color correction can be used to more accuratelyidentify and associate color samples taken under unknown lightingconditions with color names of colors captured under known lightingconditions.

Comparing Sets of Colors in an Automatically Calibrated Environment

While FIG. 7B only shows a single color sample 780 for comparison withthe color sample 773, a set of color samples captured under knownlighting conditions may be compared with the color sample 773. The setof color samples would also be captured with a color reference bar 770under known lighting conditions such as under the spectrophotometer 777.

The head mounted display device 600 can then be used to capture thecolor of the color reference bar 770 and color sample 773 under unknownlighting conditions, calculate the inverse transform matrix, apply acolor correction to the color 773A of the color sample 773 obtaining thecorrected color 773B, and compare the corrected color 773B with theknown set of colors of the color samples captured in a calibratedenvironment with known lighting conditions.

Detecting Color Gradients Over Time

A number of applications of color comparison can benefit from observingcolor changes over time. Color changes over time can be recorded usingthe camera and video capabilities of the head-mounted-display. Thecamera records a temporal sequence of images. The processor can extracta color of a color sample in each image and calculate a plurality ofcolor gradients (e.g. a difference in RGB color values) from one imageto the next over the sequence of images of a known time period. Thecolor gradients can be used for comparison against known color gradientsto improve the color comparison process.

Previously, the head mounted display device was issued a command toenter a color selection mode and capture a single still image of colorof a color sample. In capturing video, the head mounted display deviceis instructed to enter a different mode, a video recording mode.

The user issues a voice command/touchpad click to the head mounteddisplay device to enter into a video recording mode. A color sample ofinterest is then similarly selected as described and shown withreference to FIGS. 1A-1C. Alternatively, a set of color samples ofinterest may be similarly selected as described and shown with referenceto FIGS. 4A-4G.

A video of the color sample of interest is captured including aplurality of images over a known period of time with time stamps. Afterthe video is captured, the color of the color sample of interest may beanalyzed and determined in each image of the video. The difference incolor from one image to the next, a color gradient, may be calculated.Knowing the time stamp from one image to the next, gradients over timemay be calculated between each image.

A set of colors of color samples may then be selected for comparison,such as described with reference to FIGS. 4A-4G. A color compareoperation may then performed as was described with reference to FIGS.5A-5B.

Known color gradients may be received with or calculated from the set ofcolors of color samples. The computed color gradients of the colorsample changing over time may be compared with the known color gradientsof the set of colors of color samples. This may provide a more reliablecolor comparison.

The computed color gradients from the video of the color sample may alsoprovide information associated with a start time and an end time ofcolor change of the color sample, such as a start and stop time of achemical reaction. If there is no change in color gradient near thebeginning of the video, the chemical reaction may not have started. Ifthere is no change in color gradient near the end of the video, thechemical reaction may have stopped.

Methods and apparatus described herein may be used in applicationsrelated to the food industry. For example, a cake may be observed whileit is baking to determine when it is fully baked to avoid under cookingand over cooking. Liquids with different colors can be observed whenbeing mixed together (e.g., making a kir royal drink) so that the properconcentrations of each is made. Methods and apparatus described hereinmay be used in applications related to the medical industry. Bodily andbiologic samples may be observed to extract colorimetric information.For example, wound treatments, skin color changes, and urinalysis colorchanges can be analyzed for color changes to determine medical conditionchanges. Methods and apparatus described herein may be used inapplications related to the photography industry. For example, methodsand apparatus described herein may be used to detect color changes inthe sun light to capture the best light at sunset.

Comparing colors taken in uncontrolled lighting conditions can be usedto perform color corrections, establish color calibrations, generatecolor trajectories, measure the illumination of scenes, (delivering luxmeter measurements in uncontrolled lighting conditions), measure colorgradients over time, correct for color reflection, and correct fortextured supports.

Applications

The application of automatic color comparison with head-mounted displaysor augmented reality glasses include those in photometry, colorimetry,and reagent interpretation. The generic photometry applications includedetection of illumination (e.g., a lux meter), detection of colorbalance, and detection of color variations (such as at or aroundsunset). Specific colorimetry applications include color matching (e.g.,application to guide paint choice, colors in frames, buying clothes,etc.); color interpretation for color blind people; color determinationof textured objects; color classification of textured objects; colormatching of textured objects; and color gradient over time.

Referring now to FIG. 8 , a head-mounted display device or augmentedreality glasses executing an automatic color comparison application canbe used in the interpretation of reagent dipsticks. FIG. 8 illustrates areagent dipstick 880 and a reference color chart 881. The reagentdipstick 880 may be an off-the-shelf reagent dipstick.

The reagent dipstick 880 includes one or more reagent test pads 850,each with a reagent to analyze an analyte in a biological sample. Theone or more reagent test pads 850 undergo a chemical reaction whenexposed to the analytes in a biological sample. In response to thechemical reaction, the color of the reagent test pads change over timein response to concentrations or levels of analyte in the biologicalsample. The final color of the reagent test pad shortly after thechemical reaction is completed is desirable to determine theconcentration or level of analyte in the biological sample.

After being exposed to the biological sample, the dipstick 880 may beplaced next to the color chart 881. A video of the chart 881 and thedipstick 881 may be captured by the camera in the head-mounted displaydevice to capture the reagent test pads changing color over time. Thecolor of the reagent test pads may be analyzed and its color levelscalculated for each frame by the processor in the head-mounted display.The colors 885 in the reference chart may also be analyzed by theprocessor with color values assigned to each reference color sample inthe set 885. A color comparison may be performed between the colorvalues of the reagent test pads and the color values of set of referencecolors 885 of the chart 881, on a frame by frame basis if desired. Afinal stable color of the reagent test pads in an image frame,representing an end of a chemical reaction, is desirable to compare withthe reference colors of color samples in the color chart 881.

A gradient of the color change of the reagent test pads may becalculated between image frames by the processor. A known gradient maybe computed from the chart 881 for each test pad 850. A set of colors883 for a given reagent test pad 882 may be selected and a knowngradient computed by the processor. The processor may further comparethe known gradient from the set of reference colors 883 to the gradientcomputed for the reagent test pad 882.

Analytical and/or statistical methods may be used by the processor onthe colors of the reagent test pads and reference colors 881 captured bythe camera (and optionally with scene information provided by thecamera) in order to determine the nearest final color of reagent toreference color in the set and the corresponding analyte levels in thebiological fluid being tested.

Images of the reagent dipstick 880 and the reference color chart 881 aretypically captured under the same lighting condition, such that autocolor correction for different lighting conditions is unnecessary.However, if the images of the reagent dipstick 880 were captured underdifferent lighting conditions from that of the colors in the referencecolor chart 881, it may be desirable to automatically correct for colordifferences to improve the accuracy of the color comparison process andultimately the prediction of analyte concentration in a biologicalsample.

To gain further accuracy in the color comparison process, the reagentdipstick 880 and/or the color chart 881 may include a color referencebar 770 such as shown in FIG. 8 , With the color reference bar 770, anautomated color correction process can occur prior to the comparison ofcolor and gradients.

Interpretation of Scanaflo Tests

Referring now to FIG. 9 , a test paddle 900 is illustrated including acolor reference bar 770, a matrix or two-dimensional bar code 910, and aset of reagent test pads 920.

Each reagent test pad includes a reagent that can chemically react withan analyte in a biological sample. In response to the chemical reaction,the color of the reagent test pads change over time in response toconcentrations or levels of analyte in the biological sample. The finalcolor of the reagent test pad shortly after the chemical reaction iscompleted is desirable to determine the concentration or level ofanalyte in the biological sample.

A single image of the final color of the reagent test pads in the setmay be captured, However, for a more accurate analysis and result, aseries of images over the chemical reaction time of the reagent testpads is desirable to capture in a video using the camera in thehead-mounted display.

After being exposed to the biological sample, a video of the testpaddle, including the color bar and the set of test pads 920 may becaptured by the camera in the head-mounted display. The video captures atemporal sequence of images of the reagent test pads changing color overtime. The color of the reagent test pads may be analyzed and its colorlevels calculated for each frame by the processor in the head-mounteddisplay. A gradient of the color change of the reagent test pads may becalculated between image frames by the processor.

The captured colors of a reagent test pad 991 in the video images are tocompared to a set of calibration curves. The sets of calibration curvesrepresent the colors of a test pad corresponding to the whole spectrumof analyte concentrations, at then end of the reaction, through absolutecalibration.

A final stable color of the reagent test pads in an image frame,representing an end of a chemical reaction, is desirable to compare withthe set of calibration curves to determine the analyte concentration orlevel. However, the images of the set of reagent test pads 920 arecaptured under different lighting conditions from that of the colors inthe set of calibration curves. Information regarding the calibrationcurves, lighting conditions, and colors of the color reference bar maybe obtained over the internet by using the two dimensional bar code 910.With the lighting conditions of the calibration curves, standard colorsof the color reference bar, and captured color of the color referencebar 770, an inverse transform matrix may then be computed by theprocessor to correct the captured colors of the reagent test pads. Thecaptured colors of the reagent tests pads are color corrected by theprocessor using the computed inverse transform matrix. The result is thenearest color in the automatically calibrated environment, as describedin Burg '397.

As mentioned previously, a gradient of the color change of the reagenttest pads may be calculated between image frames by the processor. Burg'536 introduces an additional method for analyte interpretation based onthe change of color gradients corresponding to the chemical kinetics ofthe reaction, typically described in the art by the Michaelis-Mentonequation. This method can increase precision because it bases itsresults on a video-sequence of images versus a single image.

Augmented Reality Glasses

Referring now to FIG. 10 , augmented reality glasses 1000 are shown. Theaugmented reality glasses 1000 include a memory 1008 and a processor1006 coupled together. A camera 1004 coupled to the processor is used tocapture images. A small display device 1002 coupled to the processor1006 is located in one eyepiece. The other eyepiece has an eyeglass orlens 1010 that may be transparent to allow the user to see a real fieldof view. Preferably, the camera 1004 is mounted to or integral with theglasses 1000, however some Heads-Up-Display (HUD) devices or headmounted display devices may not include an integrated camera. In suchcases, another image capture device connected to the processor may beused to capture images in front of the user in his/her field of view. Inan alternate embodiment, the display device 1002 is substituted by alens 1010′ that can receive a projected image from a projecting device1050 mounted to a temple 1030 of the eyeglass frame and coupled to theprocessor 1006.

Enhancing Street Signs

In FIG. 10 , the augmented reality glasses 1000 can be used to augmentreality while operating or riding in a vehicle. In this application,street signs are extracted from images, enhanced, and displayed in thedisplay device 1002 in the vision of users wearing the augmented realityglasses 1000. Street signs have high visibility colors with recognizableshapes that can be detected in images and extracted so that theinformation is enhanced to the user.

The eyeglass 1010 of the glasses 1000 shown in FIG. 10 illustrate a realstreet view 1020 as perceived by the eye of the user. The real streetview 1020 includes a road with street signs 1021 near the edge of theroad.

The display device 1002 in the other side of the eyeglass shows theimage of the street captured by the camera 1004 but augmented withdigitally created street signs 1022 to form an augmented street view1025. In one embodiment, the color-coded street signs in the image arerecognized by the processor, extracted from the image, magnified insize, and temporarily overlaid onto the image of the street, as thedigitally created street signs 1022, for display in the display device1002. After the vehicle passes the signs 1021, the digitally createdstreet signs 1022 are removed from the street images displayed in thedisplay device 1002.

Enhancing Color Documents

Referring now to FIG. 11 , the augmented reality glasses 1000 withapplication software can be used to enhance the reading of color maps. Acolor in a color map may be enhanced to display more relevantinformation with emphasis. For example, a route with a yellow color maybe detected in the color map 1102 representing the route taken throughstations of a transportation system. The yellow colored route may beenhanced in a manner to emphasize the route, overlaid onto an image ofthe map, and displayed in the display device 1002.

The eyeglass 1010 in FIG. 11 illustrates the original map 1102 asperceived by one eye of the user through the eyeglass. The camera 1004captures an image of the map 1102 and displays an enhanced color mapimage 1104 in the display device 1002 of the opposite eyeglass of theglasses 1000. The enhanced color map image 1104 includes an enhancedyellow color route 1114 to enhance a route that may be of more interestto a user.

The application enhances the color map 1102 by emphasizing a particularcolor in the enhanced color map image 1104. Enhancing a color map canassist people having difficulties reading a map. Moreover, generallyenhancing color information in any document such as with an emphasizedor enhanced color can assist people that have impaired color vision.

Control Processes

Color changes sometimes occur over a process or method of preparation ofa good, such as food or baked goods. Measuring the color change can helpcontrol—baking, roasting, torrefying—the speed at which food is preparedor cooked. For example, when using a broiler or a high temperature oven,the colors of goods in the oven first evolve slowly before acceleratingexponentially. Bakers may use their skill, precise thermometers, and/ortimers/stop watches to gage the doneness of a baked good, for example.

Referring now to FIG. 12 , the augmented reality glasses 1000 may beused with software to assist the gage of doneness of baked goods orother foods that are cooked. The augmented reality glasses 1000 withapplication software may track the speed of color evolution of bakedgoods or other food. The camera 1004 of the augmented reality glasses1000 captures video of the baked goods as its changes color. Theprocessor can measure the color gradient of the changing color to allowfor dynamic adjustments in the baking or cooking process to get thedesired result of doneness. For example, when baking croissants the oventemperature may be raised to achieve a desired color gradient over time.

FIG. 12 illustrates a baked good (or cooked food) 1202 with a currentcolor at a give time through the eyeglass 1010. The camera captures animage of the baked good with its current color at the given time anddisplays it in the display device 1002. The processor analyzes thecurrent color of the baked good captured in the image. Overlaid onto thecaptured images of the baked good is a color gradient chart 1210 thatincludes a color gradient curve 1214. The color gradient curve 1214represents the goal of the baking/cooking process for the selected bakedgood/cooked food. The color gradient curve 1214 represents how the bakedgood/cooked food should be baked or cooked over time.

The processor plots the current color of the baked good/cooked food asan arrow 1212 at the current time on the time line of the color gradientchart 1210. The end point of the arrow head of the arrow 1212 mayrepresent the measure of color in the current baked good. If the endpoint of the arrow 1212 is below the color gradient curve 1214, thetemperature may be increased or the baking time may be increased toobtain the desired color goal and doneness in the baked good/cookedfood. Assuming the temperature is to remain the same, the processor maycalculate and display the remaining baking time or cooking time. if theend point of the arrow 1212 is above the color gradient curve 1214, thetemperature may be decreased or the baking time may be decreased toobtain the desired color goal and doneness in the baked good/cookedfood.

In this manner, the glasses 1000 augment reality of the baking/cookingprocess by adding a color gradient chart 1210 and arrow 1212 in theaugmented baked good image 1204.

CONCLUSION

When implemented in software, the elements of the embodiments areessentially the code segments or instructions executable by a processor(e.g., processor 1006 shown in FIGS. 10-12 ) to perform the necessarytasks. The program or code segments can be stored in a storage device ora processor readable medium (e.g., memory 1008 shown in FIGS. 10-12 ).Examples of a processor readable medium include an electronic circuit, asemiconductor memory device, a read only memory (ROM), a flash memory,an erasable programmable read only memory (EPROM), a floppy diskette, aCD-ROM, an optical disk, a hard disk, a fiber optic medium, a radiofrequency (RF) link, etc. The code segments or instructions may bedownloaded via computer networks such as the Internet, Intranet, etc.

While this specification includes many specifics, these should not beconstrued as limitations on the scope of the disclosure or of what maybe claimed, but rather as descriptions of features specific toparticular implementations of the disclosure. Certain features that aredescribed in this specification in the context of separateimplementations may also be implemented in combination in a singleimplementation. Conversely, various features that are described in thecontext of a single implementation may also be implemented in multipleimplementations, separately or in sub-combination. Moreover, althoughfeatures may be described above as acting in certain combinations andeven initially claimed as such, one or more features from a claimedcombination may in some cases be excised from the combination, and theclaimed combination may be directed to a sub-combination or variationsof a sub-combination. Accordingly, the claimed embodiments should belimited only by patented claims that follow below.

1-46. (canceled)
 47. A method comprising: selecting a first color samplewithin a first target area in a first image of a first object, the firstcolor sample being a color of a reagent test pad; selecting a pluralityof color samples within a second target area in a second image of asecond object, the plurality of color samples being configured to enableinterpretation of a urinalysis, wherein the first color sample isilluminated in lighting conditions different from lighting conditions ofthe plurality of color samples; with at least one processor, using aninverse transform matrix to compare the first color sample against theplurality of color samples to determine at least one of a measure ofcolor difference or a measure of color equivalence between the firstcolor sample of the first object and the plurality of color samples ofthe second object, wherein the inverse transformation matrix links theplurality of color samples under first lighting conditions tocorresponding color samples under second lighting conditions; and usingresults of the comparing to interpret the urinalysis by determininganalyte level in a biological fluid being tested.
 48. The method ofclaim 47, wherein the display device is a part of a head-mounted displaydevice.
 49. The method of claim 47, wherein the results displayed to theuser in the display device include the first color sample, a first colorname and first color values associated with the first color sample; thesecond color sample, a second color name and second color valuesassociated with the second color sample; and difference color values toindicate the measure of color difference between the first color sampleand the second color sample.
 50. The method of claim 47, wherein theresults displayed to the user in the display device include the firstcolor sample, a first color name and first color values associated withthe first color sample; the second color sample, a second color name andsecond color values associated with the second color sample; andequivalence color values to indicate the measure of color equivalencebetween the first color sample and the second color sample.
 51. A methodcomprising: selecting a first color sample under a target in a firstimage of a first object displayed in a display device; selecting aplurality of color samples having a plurality of different colors as aselected set of colors; with a processor, comparing color of the firstcolor sample against the plurality of different colors of the selectedset of colors to determine a closest match color sample of color in theselected set of colors and measure a color difference between the colorof the first color sample and the color of the closest match colorsample; and displaying the results of the color comparison to a user inthe display device.
 52. The method of claim 51, wherein the displaydevice is a part of a head-mounted display device.
 53. The method ofclaim 51, wherein the results displayed to the user in the displaydevice include the first color sample, a first color name and firstcolor values associated with the first color sample; the plurality ofcolor samples including the closest match color sample; a second colorname and second RGB color values associated with the closest match colorsample; and difference RGB color values to indicate the difference incolor between the first color sample and the closest match color sample.54. The method of claim 53, wherein the results displayed to the user inthe display device include an emphasis device to emphasize the closestmatch color sample in the plurality of color samples.
 55. The method ofclaim 54, wherein the emphasis device is one of a color ring around theclosest match color sample or a bulls eye around the closest match colorsample.
 56. An apparatus comprising: a display device in a head-mounteddisplay device displaying a color comparison results window including afirst color sample within a target area of a first object, a secondcolor sample within a target area of a second object, first and secondcolor names associated with the first and second color samples, firstand second color values associated with the first and second colorsamples, and difference color values to indicate the difference in colorbetween the first color sample and the second color sample.
 57. Theapparatus of claim 56, wherein the display device in the head-mounteddisplay device further displays a first color selection window includinga target, the first object under the target, and user interface textinforming the user regarding the selection of the first color samplewithin the target area of the first object.
 58. The apparatus of claim57, wherein the display device in the head-mounted display devicefurther displays a second color selection window including the target,the second object under the target, and user interface text informingthe user regarding the selection of the second color sample within thetarget area of the second object.
 59. The apparatus of claim 58, whereinthe target is a sight of a bulls-eye or a cross-hair.
 60. The apparatusof claim 58, wherein the first object is a reagent dipstick, and thesecond object is reference color chart.